What's in a Name? Experimental Evidence of Gender Bias in Recommendation Letters Generated by ChatGPT.

BACKGROUND: Artificial intelligence chatbots such as ChatGPT (OpenAI) have garnered excitement about their potential for delegating writing tasks ordinarily performed by humans. Many of these tasks (eg, writing recommendation letters) have social and professional ramifications, making the potential social biases in ChatGPT's underlying language model a serious concern. OBJECTIVE: Three preregistered studies used the text analysis program Linguistic Inquiry and Word Count to investigate gender bias in recommendation letters written by ChatGPT in human-use sessions (N=1400 total letters). METHODS: We conducted analyses using 22 existing Linguistic Inquiry and Word Count dictionaries, as well as 6 newly created dictionaries based on systematic reviews of gender bias in recommendation letters, to compare recommendation letters generated for the 200 most historically popular "male" and "female" names in the United States. Study 1 used 3 different letter-writing prompts intended to accentuate professional accomplishments associated with male stereotypes, female stereotypes, or neither. Study 2 examined whether lengthening each of the 3 prompts while holding the between-prompt word count constant modified the extent of bias. Study 3 examined the variability within letters generated for the same name and prompts. We hypothesized that when prompted with gender-stereotyped professional accomplishments, ChatGPT would evidence gender-based language differences replicating those found in systematic reviews of human-written recommendation letters (eg, more affiliative, social, and communal language for female names; more agentic and skill-based language for male names). RESULTS: Significant differences in language between letters generated for female versus male names were observed across all prompts, including the prompt hypothesized to be neutral, and across nearly all language categories tested. Historically female names received significantly more social referents (5/6, 83% of prompts), communal or doubt-raising language (4/6, 67% of prompts), personal pronouns (4/6, 67% of prompts), and clout language (5/6, 83% of prompts). Contradicting the study hypotheses, some gender differences (eg, achievement language and agentic language) were significant in both the hypothesized and nonhypothesized directions, depending on the prompt. Heteroscedasticity between male and female names was observed in multiple linguistic categories, with greater variance for historically female names than for historically male names. CONCLUSIONS: ChatGPT reproduces many gender-based language biases that have been reliably identified in investigations of human-written reference letters, although these differences vary across prompts and language categories. Caution should be taken when using ChatGPT for tasks that have social consequences, such as reference letter writing. The methods developed in this study may be useful for ongoing bias testing among progressive generations of chatbots across a range of real-world scenarios. TRIAL REGISTRATION: OSF Registries osf.io/ztv96; https://osf.io/ztv96.

Engaging Cancer Care Physicians in Off-Label Drug Clinical Trials: Human-Centered Design Approach.

BACKGROUND: Using a human-centered design (HCD) approach can provide clinical trial design teams with a better understanding of the needs, preferences, and attitudes of clinical trial stakeholders. It can also be used to understand the challenges and barriers physician stakeholders face in initiating and completing clinical trials, especially for using off-label drugs (OLDs) to treat unmet clinical needs in cancer treatment. However, the HCD approach is not commonly taught in the context of clinical trial design, and few step-by-step guides similar to this study are available to demonstrate its application. OBJECTIVE: This study aims to demonstrate the feasibility and process of applying an HCD approach to creating clinical trial support resources for physician stakeholders to overcome barriers to pursuing clinical trials for OLDs to treat cancer. METHODS: An HCD approach was used to develop OLD clinical trial support concepts. In total, 45 cancer care physicians were contacted, of which 15 participated in semistructured interviews to identify barriers to prescribing OLDs or participating in cancer OLD clinical trials. Design research is qualitative-it seeks to answer "why" and "how" questions; thus, a sample size of 15 was sufficient to provide insight saturation to address the design problem. The team used affinity mapping and thematic analysis of qualitative data gathered from the interviews to inform subsequent web-based co-design sessions, which included creative matrix exercises and voting to refine and prioritize the ideas used in the final 3 recommended concepts. RESULTS: The findings demonstrate the potential of HCD methods to uncover important insights into the barriers physicians face in participating in OLD clinical trials or prescribing OLDs, such as recruitment challenges, low willingness to prescribe without clinical data, and stigma. Notably, only palliative care participants self-identified as "frequent prescribers" of OLDs, despite high national OLD prescription rates among patients with cancer. Participants found the HCD approach engaging, with 60% (9/15) completing this study; scheduling conflicts caused most of the dropouts. Over 150 ideas were generated in 3 co-design sessions, with the groups voting on 15 priority ideas that the design team then refined into 3 final recommendations, especially focused on increasing the participation of physicians in OLD clinical trials. CONCLUSIONS: Using participatory HCD methods, we delivered 3 concepts for clinical trial support resources to help physician stakeholders overcome barriers to pursuing clinical trials for OLDs to treat cancer. Overall, integrating the HCD approach can aid in identifying important stakeholders, such as prescribing physicians; facilitating their engagement; and incorporating their perspectives and needs into the solution design process. This paper highlights the process, methods, and potential of HCD to improve cancer clinical trial design. Future work is needed to train clinical trial designers in the HCD approach and encourage adoption in the field.

A Web-Based Intervention to Address Risk Factors for Maternal Morbidity and Mortality (MAMA LOVE): Development and Evaluation Study.

Background: Maternal mortality in the United States is a public health crisis and national emergency. Missed or delayed recognition of preventable life-threatening symptoms and untimely treatment of preventable high-risk medical conditions have been cited as key contributors to the nation's worsening mortality rates. Effective strategies are urgently needed to address this maternal health crisis, particularly for Black birthing populations. Morbidity and Mortality Assessment: Lifting Outcomes Via Education (MAMA LOVE) is a web-based platform that focuses on the identification of maternal morbidity and mortality risk factors. Objective: The purpose of this paper is to present the conceptualization, development, heuristics, and utility evaluation of the web-based maternal mortality risk assessment and educational tool MAMA LOVE. Methods: A user-centered design approach was used to gain feedback from clinical experts and potential end users to ensure that the tool would be effective among groups most at risk for maternal morbidity and mortality. A heuristic evaluation was conducted to evaluate usability and need within the current market. Algorithms describing key clinical, mental health, and social conditions were designed using digital canvas software (Miro) and incorporated into the final wireframes of the revised prototype. The completed version of MAMA LOVE was designed in Figma and built with the SurveyJS platform. Results: The creation of the MAMA LOVE tool followed three distinct phases: (1) the content development and creation of an initial prototype; (2) the feedback gathering and usability assessment of the prototype; and (3) the design, development, and testing of the final tool. The tool determines the corresponding course of action using the algorithm developed by the authors. A total of 38 issues were found in the heuristic evaluation of the web tool's initial prototype. Conclusions: Maternal morbidity and mortality is a public health crisis needing immediate effective interventions. In the current market, there are few digital resources available that focus specifically on the identification of dangerous symptoms and risk factors. MAMA LOVE is a tool that can address that need by increasing knowledge and providing resources and information that can be shared with health care professionals.

Pivot, persevere, or perish: how Ellume Health overcame development and regulatory obstacles to become the first authorized over-the-counter COVID-19 test in the United States.

The Ellume COVID-19 home test from Ellume Health (Brisbane, Aus) became the first COVID-19 diagnostic tool authorized for home use by the United States FDA in December 2020. This early pandemic success was built on over ten years of work on the Ellume influenza home test. Ellume overcame critical technology challenges during the development of their influenza test. In addition, it faced a recall of its COVID-19 home test in 2021 due to false positive results. While Ellume initially persevered through the recall and restarted sales in the United States, the combination of the recall and the wide availability of competitors' low-cost over the counter tests in the United States led to Ellume entering voluntary administration in September 2022. This paper traces the history of Ellume and how 10 years of experience with a home influenza test allowed the company to quickly develop the Ellume COVID-19 home test. We will also provide to diagnostic developers the key strategies employed by Ellume to succeed while highlighting the pitfalls that have challenged the company's business success.

Designing for simplicity: lessons from Mesa Biotech for microfluidic entrepreneurs and early-stage companies.

The COVID-19 pandemic has proven the need for point-of-care diagnosis of respiratory diseases and microfluidic technology has risen to the occasion. Mesa Biotech (San Diego, CA) originally developed the Accula platform for the diagnosis of influenza A and B and then extended the platform to SARS-CoV-2. Mesa Biotech has experienced tremendous success, culminating in acquisition by Thermo Fisher for up to $550m USD. The Accula microfluidics platform accomplished the leap from the lab to commercial product through clever design and engineering choices. Through information obtained from interviews with key Mesa Biotech leaders and publicly-available documents, we describe the keys to Mesa's success and how they might inform other lab-on-a-chip companies.